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(57) ABSTRACT 

An apparatus an method are disclosed for dividing a two- 
dimensional matrix of pixels into at least first and second 
areas. For each pixel, it is determined in which the area the 
pixel is located. Then, the pixels in the first area are 
processed according to a first processing/compression 
scheme, and the pixels in the second area are processed 
according to a second processing/compression scheme. A 
data bit stream is generated which represents the processed/ 
compressed pixels and their associated areas, and the data bit 
stream can be stored or transmitted in lieu of the entire 
matrix of pixels. If desired, the pixels in the first area can be 
processed less frequently than the pixels in the second area, 
and the areas themselves can be established in response to 
the frequency of variations in the pixel values. 

21 Claims, 8 Drawing Sheets 
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APPARATUS AND METHOD FOR 
PROCESSING VIDEO DATA 

This application is a continuation, of application number 
08/266,907, filed Jun. 27, 1994 now abandoned. 5 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates generally to video data 10 
processing, and more particularly to efficiently transmitting 
and storing video data. 

2. Description of the Related Art 

The area of multimedia is a large and still-growing 
industry which stores, processes, and transmits data and ^ 
programs in multimedia formats. Multimedia programming 
includes, among other things, educational and entertainment 
audio-visual programs. Because of the world-wide accep- 
tance of multimedia programming, and the demand for ever 
more such programming, the multimedia industry has 20 
launched several large-scale efforts geared toward providing 
multimedia programming on demand. 

Most commonly, a multimedia program consists of both 
audio data and visual data. With current technology, video 
data is generated by filming an object and generating a series 25 
of raster-scanned video images of the object. Each image is 
established by a two-dimensional matrix of picture 
elements, referred to as "pixels", and the data record estab- 
lished by each matrix is relatively large, i.e., the data record 
established by each matrix contains a relatively large num- 30 
ber of pixels. 

In many approaches common in the art, each pixel defines 
a numerical value (currently an integer in the range 0-255) 
which represents a color or a color and an intensity. To 35 
transmit or store an image, a bit stream representing all of 
the pixels in the matrix and their associated values is 
generated and then stored or transmitted as appropriate. 

Typically, when a multimedia program includes a video 
display, about thirty such images of an object must be 40 
generated and processed for each second of viewing time. It 
accordingly can be appreciated that a single video program 
requires the storage and/or transmission of a large number of 
relatively large data records, on the order of tens of megabits 
per second. Indeed, according to the May -June 1992 issue of 45 
the MIT Technology Review, "video is a glutton for band- 
width" which consequently can "saturate" many existing 
networks, particularly in the event that a single network 
must handle more than a few video transmissions. Such a 
consideration is of growing importance in light of the advent 50 
of video-on-demand discussed above. 

Accordingly, to reduce the amount of a multimedia pro- 
gram's video data which must be stored and/or transmitted, 
methods have been introduced for "compressing" video 
data. Data compression essentially omits from the bit stream 55 
that is to be transmitted certain data in a particular data 
record. For example, in video image transmission, a simple 
compression technique may include, in the bit stream to be 
transmitted, only data which represents pixels that have 
changed relative to the immediately preceding image frame. 60 
Many other compression techniques exist which compress 
data to relatively lesser or greater degrees. 

As the skilled artisan will appreciate, a trade-off exists in 
determining how much to compress video data. More 
particularly, it is the case that relatively great compression 65 
results in less data to transmit, but also tends to undesirably 
result in corruption of the image that is received for display. 
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Nonetheless, current compression techniques ordinarily 
are not tailored to a particular program or application. 
Likewise, current compression techniques do not permit a 
user to modify or otherwise tailor the compression process- 
ing to a particular program or application. 

This is unfortunate, because a compression technique that 
may be suitable for one program may not be suitable for 
another. For example, a technique which compresses data to 
a relatively large degree may be suitable for a program in 
which relatively little on-screen activity occurs, but may be 
unsuitable for a program having a large amount of on-screen 
activity, because a large degree of data compression could 
corrupt the video image in such a program an unacceptable 
amount. Likewise, a technique that compresses data a rela- 
tively small degree may be suitable for a program in which 
much on-screen activity occurs, but would be an inefficient 
technique for a program having a small amount of on-screen 
activity, because much data would be unnecessarily included 
in the bit stream. Further, it is often the case in video 
programs that not all on-screen activity is equally important, 
yet the entire image typically is subjected to a single 
compression technique. As can be inferred from the above 
discussion, this can result in either unacceptable corruption 
of important portions of the image or inefficient transmission 
of bit streams which include more data than is necessary. 

Accordingly, it is an object of the present invention to 
provide a system for interactively establishing one or more 
compression techniques for a pixel matrix. Another object of 
the present invention is to provide a system for dynamically 
establishing a compression technique for a pixel matrix. Still 
another object of the present invention is to provide a 
method and apparatus for establishing compression tech- 
niques which is easy to use and cost-effective. 

SUMMARY OF THE INVENTION 

A computer system for processing a video image includes 
a central processing unit (CPU) and at least one data record 
representative of a video image frame. The data record, 
which is accessible by the CPU, includes a plurality of 
elements that represent a plurality of pixels arranged in a 
matrix. 

A boundary de finer establishes first and second areas of 
the matrix, and a locator is operably engaged with the 
boundary definer for determining, for at least some of the 
pixels, which of the pixels are located in the first area and 
which of the pixels are located in the second area. A pixel 
processor is operably engaged with the locator for process- 
ing the elements representing the pixels in the first area in 
accordance with a first processing scheme and for process- 
ing the elements representing the pixels in the second area 
in accordance with a second processing scheme. 

Preferably, the pixel processor generates a bit stream 
which is representative of the pixels and the area of each. 
Also, the system preferably includes an I/O device for 
selectively causing the bit stream to be stored in a storage 
device or transmitted across a communications network. If 
desired, the bit stream representing the pixels in the first area 
is compressed relative to the number of pixels in the first 
area. 

In one embodiment of the present invention, the video 
program defines a sampling cycle, and the system further 
includes a timer for establishing a sampling interval thresh- 
old which is longer than the sampling cycle. As intended by 
the present invention, the pixel processor processes the 
elements representing the pixels in the second area each 
sampling cycle, and the pixel processor processes the ele- 
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ments representing the pixels in the first area each sampling 
interval threshold. Additionally, each pixel defines a corre- 
sponding variable value, and the boundary definer can 
establish the first and second areas in response to variations 
in the values of the pixels. 

In another aspect of the present invention, an apparatus 
for processing data elements representing an image includes 
means for establishing first and second areas of the image, 
and means for determining, for at least some of the elements, 
which of the elements are located in the first area and which 
of the elements are located in the second area. Also, the 
apparatus includes means for processing the elements in the 
first area in accordance with a first processing scheme and 
for processing the elements in the second area in accordance 
with a second processing scheme. 

In still another aspect of the present invention, a method 
is disclosed for processing data representing pixels arranged 
in a matrix. The method of the present invention includes 
defining at least first and second areas within the matrix and 
determining which pixels are within the first area and which 
pixels are within the second area. Then, the data representing 
pixels in the first area is processed in accordance with a first 
processing scheme, while the data representing pixels in the 
second area is processed in accordance with a second 
processing scheme. 

In yet another aspect of the present invention, a system for 
processing pixels of a video image which defines a sampling 
cycle includes a data record. The data record in turn includes 
a matrix of pixels which represent an image, and the matrix 
is defined by at least first and second areas. Also, a timer is 
provided for establishing a sampling interval threshold 
which is longer than the sampling cycle. Furthermore, a 
pixel processor processes the pixels in the first area each 
sampling cycle and processes the pixels in the second area 
each sampling interval threshold. 

In another aspect of the present invention, a method for 
processing pixels includes the steps of providing a data 
record that includes a matrix of pixels representing an 
image, wherein the matrix is defined by at least first and 
second areas. A sampling cycle and a sampling interval 
threshold are established such that the sampling interval 
threshold is longer than the sampling cycle, and the pixels in 
the first area are processed each sampling cycle. On the other 
hand, the pixels in the second area are processed each 
sampling interval threshold. 

In yet another aspect of the present invention, a method 
for processing pixels which are arranged in a two- 
dimensional matrix includes the steps of establishing first 
and second areas of the matrix, and then determining, for at 
least some of the pixels, which of the pixels are located in 
the first area and which of the pixels are located in the 
second area. Next, the pixels in the first area are processed 
in accordance with a first processing scheme, while the 
pixels in the second area are processed in accordance with 
a second processing scheme. 

The details of the present invention, both as to its structure 
and operation, can best be understood with reference to the 
accompanying drawings, in which like reference numerals 
refer to like parts, and in which: 

BRIEF DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

FIG. 1 is a perspective view of the apparatus for process- 
ing video data of the present invention; 

FIG. 2 is a schematic diagram of a bit stream of the 
present invention; 
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FIG. 3 is a flow chart of the overall method for processing 
video data of the present invention; 

FIG. 4 is a flow chart of the auto focus boundary method 
for automatically determining area boundaries; 

FIG. 5 is a flow chart of the method for locating pixels; 

FIG. 6 is a flow chart of the pixel sampling suppression 
method; and 

FIG. 7 is a flow chart of the auto focus sampling rate 
| method for automatically determining the sampling interval 
threshold. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 

Referring initially to FIG. 1, a computer system for 
processing video program data is shown, generally desig- 
nated 10. As shown in FIG. 1, the system 10 includes a 
monitor 12 for presenting a video display of an image 14 of 
an object, e.g., a person's face. In accordance with principles 
( well-known in the art, the video monitor 12 is a raster- 
scanned monitor which includes a matrix of picture 
elements, referred to herein as "pixels", 16, and the pixels 16 
establish the image 14 which is presented on the video 
display 12. Although not ordinarily individually detectable 
. by the human eye, for clarity of disclosure the pixels 16 are 
shown in FIG. 1, and the spacing between adjacent pixels 16 
shown in FIG. 1 is greatly exaggerated. 

Typically, about thirty (30) image frames per second of 
the video program to be processed are generated. As 

3 0 intended by the present invention, a sampling cycle is 
defined to be the inverse of the number of frames per second 
of the video program. 

In accordance with the present invention, the matrix of 
pixels 16 includes a first area 18 and a second area 20. It is 

35 to be understood that the matrix of pixels 16 can be divided 
into a greater number of areas than two, and that the areas 
18, 20 can have shapes other than quadrilateral, e.g., each 
area 18, 20 can be circularly-shaped or have a polygonal 
shape other than quadrilateral. 

40 FIG. 1 shows that the system 10 includes a computer 
central processing unit (CPU) 22. As the skilled artisan will 
recognize, both still images and motion video images can be 
received by the CPU 22 for processing. Additionally, a video 
processor card 24 (shown in phantom in FIG. 1) is prefer- 

45 ably operably associated with the CPU 22 for digitizing the 
video image from camera 35. Further, an optical data storage 
device 26 and a magnetic data storage device 28 can be 
electrically connected to the CPU 22 for storing records of 
video images. Computer system 10 also includes memory 

50 13, which is a random access memory sufficiently large 
enough to hold the necessary programming and data struc- 
tures. While memory 13 is shown as a single entity, it should 
be understood that memory 13 may in fact comprise a 
plurality of modules, and that memory can exist at multiple 

55 levels, from high-speed registers and caches to lower speed 
but larger DRAM chips. Memory 13 contains control pro- 
gram 15, which comprises a plurality of machine instruc- 
tions that execute on CPU 22 to carry out this invention as 
described in more detail in the flowcharts of FIGS. 3-7. The 

60 contents of memory 13 can be loaded to and stored from 
magnetic data storage device 28 as needed. In the preferred 
embodiment, control program 15 processes (as described 
below) the digitized video images that are output from video 
processor card 24 and displays the images on monitor 12. 

65 Alternatively, control program 15 can process digitized 
images received across a communications line from a net- 
work (not shown). After processing, the video images can be 



12/23/2003, EAST Version: 1.4.1 



US 6,501,853 Bl 



10 



15 



stored in storage devices 26 and 28 or transmitted across a 
communications network. 

Moreover, an input/output (I/O) controller 30 is electri- 
cally connected to the CPU 22 for interfacing the CPU 22 
with the video monitor 12 and with appropriate control 
devices, e.g., a keyboard 32 and a mouse 34. The CPU 22 is 
advantageously a mainframe computer or personal computer 
such as an IBM AS/400 or PS/2 computer system. Also, the 
video processor card 24, camera 35, storage devices 26, 28, 
I/O controller 30, keyboard 32, and mouse 34 can all be 
off-the-shelf components. 

y In accordance with the present invention and as disclosed 
elow, the boundaries of each of the first and second areas 
18, 20 are established, and then the locations of the pixels 16 
(typically in Cartesian coordinates) are correlated to either 
one of the areas 18, 20. Next, data elements representing the 
pixels 16 that are located in the first area 18 of the matrix of 
pixels 16 can be processed using a first compression 
algorithm, and data elements representing the pixels 16 that 
are located in the second area 20 of the matrix of pixels 16 
can be processed using a second compression algorithm. 
Although in this example two areas 18 and 20 are shown 
with two corresponding compression algorithms, it should 
be understood that more than two areas with more than two 
compression algorithms could be used. As intended by the 
present invention, the term "pixel" is used interchangeably 
to refer to both the video elements of the monitor 12 that 
establish the image 14, and the data elements of a data record 
which digitally represent the matrix of pixels 16 and which 
are generated after the pixels 16 have been digitized by the 
video processor card 24. 

The compressed pixels 16 can then be parsed by area into 
a bit stream that represents the image displayed on the video 
monitor 12. Alternatively, the pixels 16 can be parsed by 
area into a bit stream that represents the image displayed on 
the video monitor 12 prior to compression, with the bit 
stream then being subjected to compression by means well- 
known in the art. 

Z' FIG. 2 shows one example of a bit stream of the present 40 
invention. As shown in FIG. 2, taking the fifth pixel row 36 
from the top of the monitor 12, a bit stream 38 has a first field 
40 which indicates that the fields following the first field 40 
represent pixels in the first area 18 of the matrix of pixels 16. 
Asecond field 42 represents the first (i.e., left-most) pixel 16 
which is located in the fifth pixel row 36, along with the 
value of the pixel. This pixel is designated by its Cartesian 
coordinates (5,1). Likewise, third and fourth fields 44, 46 
represent the second and third pixels 16 in the fifth pixel row 

J36 and the values of each. 

A fifth field 48 functions essentially as a flag that the fields 
following the fifth field 48 represent pixels 16 which are 
located in the second area 20 of the matrix of pixels 16. A 
sixth field 50 represents the fourth pixel 16 which is located 
in the fifth pixel row 36, along with the value of the pixel. 55 
It is to be understood that the entire matrix of pixels 16 can 
be parsed into a bit stream in accordance with the principles 
disclosed above, either before or after area-by-area com- 
pression. The bit stream 38 is but one example of a data 
record that represents the pixels of a video image frame. $0 

Now referring to FIG. 3, the overall method of processing 
video data in accordance with the present invention can best 
be appreciated. In this regard, FIG. 3 represents the structure 
and the function of the control program 15 shown in FIG. 1. 
As shown in FIG. 3, the control program 15 starts and the 65 
boundaries of the areas 18, 20 of the matrix of pixels 16 are 
initially established. First, an image is retrieved for process- 
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ing (block 59), Then, at block 60, the control program 15 
determines whether the operator of the system 10 has 
selected automatic boundary definition. Such a selection can 
be made by the operator using any suitable means, e.g., by 
appropriately manipulating the keyboard 32 or mouse 34. 

If automatic boundary definition has been selected, the 
control program 15 proceeds to block 62, wherein the 
boundaries of the areas 18, 20 are determined as described 
below in reference to FIG. 4. Then, the process proceeds to 
block 64, wherein it is determined, for each pixel 16, 
whether the pixel 16 is located in the first area 18 of the 
matrix of pixels 16 or the second area 20 of the matrix of 
pixels 16, as discussed more fully below in reference to FIG. 
5. 

If, on the other hand, automatic boundary definition has 
not been selected, the process proceeds to block 66, wherein 
the boundaries of the areas 18, 20 are defmed by the operator 
of the system 10 using appropriate display control means, 
e.g., the keyboard 32, mouse 34, touch-screen display, and 
so on. In manually defining the boundaries of the areas 18, 
20, the operator may also select the shape of the areas 18, 20, 
e.g., circular, rectangular, polygonal, etc. For the display 
shown in FIG. 1, to define the rectangular areas 18, 20 
shown, the operator could, e.g., sequentially position the 
cursor of the display on each of the four comers of the 
second area 20, depressing the operating switch of the 
mouse 34 at each comer or depressing the "Enter" key of the 
keyboard 32 at each comer. Alternatively, the boundaries of 
the areas 18, 20 of the matrix of pixels 16 can be pre-defined 
during initial programming. 

From block 66, the process proceeds to block 68, wherein 
it is determined, for each pixel 16, whether the pixel 16 is 
located in the first area 18 of the matrix of pixels 16 or the 
second area 20 of the matrix of pixels 16. Thus, blocks 62 
and 66 are boundary definers, and blocks 64 and 68 are pixel 
locators. 

From block 64 or 68, the process moves to block 70, 
wherein it is determined whether the operator of the system 
10 has selected a timed sampling feature described more 
fully below in reference to FIG. 6. As described below, the 
timed sampling feature of the present invention suppresses 
processing of pixels 16 which are located in an operator- 
selected one (or both) of the areas 18, 20 for a predetermined 
number of sample cycles, referred to herein as a sampling 
interval threshold. Stated differently, pixels 16 in a selected 
one of the areas 18, 20 are sampled (and, hence, are 
processed) less frequently than pixels 16 that are located in 
the other area 20, 18. Thus, block 70 is essentially a timer for 
establishing a sampling interval threshold. 

If the timed sampling feature has not been selected at 
block 70, the process moves to blocks 72 and 74. At block 
72, the pixels 16 that are located in the first area 18 of the 
matrix of pixels 16 are processed according to a first 
processing scheme, e.g., a first compression algorithm. In 
contrast, at block 74 the pixels 16 that are located in the 
second area 20 of the matrix of pixels 16 are processed 
according to a second processing scheme, e.g., a second 
compression algorithm. Thus, blocks 72 and 74 are pixel 
processors. 

It may now be appreciated that the present invention 
envisions subjecting preselected areas of the image dis- 
played on the video monitor 12 to a greater degree of 
compression than other areas. As but one example, the pixels 
16 which establish the image 14 of the object of interest, 
shown in FIG. 1 as being located within the second area 20, 
may be subjected to relatively little compression, to thereby 
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minimize corruption of the portion of the image that is a group, and so on. As yet a third alternative, the changed 
greatest interest to the video program. On the other hand, the pixels 16 simply may be grouped together by quadrant of the 
pixels 16 that are located within the second area 18 may video monitor 12. It is to be understood that the groupings 
establish only background images, and may accordingly be may be effected using these or other grouping schemes- 
subjected to a relatively great degree of compression, to 5 depending upon, among other things, the computationa 
conserve data storage requirements and transmission band- speed of the CPU 22. 
1 — -width. From block 100, the process moves to block 102 to 
From blocks 72 and 74 the process moves to block 76, determine which changed pixel 16 in each group of changed 
wherein a bit stream representing the pixels 16, e.g., the bit pixels 16 is the focal point of that group. In one presently 
stream 38 shown in FIG. 2, can be generated by means ™ preferred embodiment, for each group the changed pixel 16 
well-known in the art. It is to be understood that the bit having the mean Cartesian coordinates for that group is 
stream 38 may be generated prior to data compression. Then, defined to be the focal point. 

at block 78, the bit stream 38 is transmitted across a Next, at block 104, boundaries are determined for areas 

communications network or stored by one or both of the data that correspond to each group of changed pixels 16. The 

storage devices 26, 28 shown in FIG. 1, as established by the 15 process then returns to FIG. 3, block 64. In the presently 

operator of the system 10. At block 79, if images remain to preferred process, each area is defined by a circle centered 

be processed, the next image of the video program being on the focal point of each group and having a radius equal 

displayed on the video monitor 12 is retrieved for processing to one-half the nominal boundary size, 

at block 59. Otherwise the program stops. Accordingly, a plurality of areas may be defined by the 

If the operator of the system 10 has selected timed 20 method disclosed above. Assuming that areas having high 

sampling at block 70, the process moves to block 82, concentrations of changed pixels 16 are of relatively great 

wherein it is determined whether the operator has also interest in the video program, each such area can be desig- 

selected the automatic sampling rate feature of the present nated as an area to be processed using relatively little 

invention, discussed in detail below in reference to FIG. 7. compression. The remaining areas would be processed using 

If so, the process moves to block 84 to determine the 25 relatively great compression. 

sampling rate, and then to block 86 to suppress pixels 16 FIG. 5 shows the operation of the pixel locator of the 

from processing as described below in reference to FIG. 6. present invention. Following start, for each pixel 16, at block 

Otherwise, the process moves directly to block 86, and 106 it is determined whether the pixel 16 under test is above 

thence to blocks 72, 74 as appropriate. ^ the boundary of the second area 20 of the matrix of pixels 

Now referring to FIG. 4, one presently preferred process 16. In other words, at block 106 it is determined whether the 

by which the boundaries of the first and second areas 18, 20 "y" coordinate of the pixel 16 is above the uppermost "y" 

of the matrix of pixels 16 are automatically defined by coordinate of the boundary of the second area 20. If it is, the 

control program 15 can be appreciated. The process starts, process proceeds to block 108 to designate the pixel 16 as 

and at block 88, for each pixel 16 it is determined whether 35 being in the first area 18. The pixel 16 is then stored with a 

the value of the pixel 16 under test has changed from its tag indicating that it is in the first area 18, or it is stored in 

value in the immediately preceding image frame. If so, the a bin designated as the first area 18 bin and the control 

pixel 16 is added to a list of changed pixels at block 90, and program 15 returns to FIG. 3, block 70. 

then the process moves to block 90 determine whether the If the test at block 106 is positive, no further pixels 16 in 

pixel 16 under test is the last pixel 16 in the image frame. 4Q the same row need be tested. Thus, the process returns to 

Also, if the value of the pixel 16 under test has not changed block 106 to next test the first pixel 16 in the row immedi- 

from its value in the immediately preceding image frame, ately following the row of the pixel 16 that had been under 

the process moves directly to block 90. test. ' 

If the test at block 90 is negative, the process moves to In contrast, if the test at block 106 is negative, the process 

block 94 to retrieve the next pixel 16, and then moves back 45 moves to block 110 to determine whether the pixel 16 under 

to block 88. Otherwise, the process moves to block 96, test is below the boundary of the second area 20 of the 

wherein the changed pixels 16 are sorted by their Cartesian matrix of pixels 16. If it is, the process moves to block 108 

coordinates. Then, at block 98, a nominal boundary size is and then returns to FIG. 3, block 70. Otherwise, the process 

determined. In one presently preferred embodiment, the moves to block 112 to calculate the left and right boundaries 

nominal boundary size is defined to be twenty percent (20%) 50 of the second area 20 for the row under test by straightfor- 1 

of the size of the area that would be established by the ward geometric means. j 

changed pixels 16, if the changed pixels 16 were arranged in Next, at block 114, it is determined whether the column of \ 

a square matrix. Alternatively, the nominal boundary size the pixel 16 under test is left of the boundary of the second 

can be defined to be some other percentage of the size of the area 20. If it is, the process moves to block 108 and then 

area that would be established by the changed pixels 16, if 5 5 returns to FIG. 3, block 70. Otherwise, the process moves to 

the changed pixels 16 were arranged in, e.g., a circular block 116 to determine whether the column of the pixel 16 

pattern. The determination of the nominal boundary size can under test is right of the boundary of the second area 20. If 

be accomplished by accessing a table which correlates it is, the process moves to block 108 and returns to FIG. 3, 

number of changed pixels 16 to nominal boundary size. block 70. Otherwise, the process moves to block 118 to 

From block 98, the process moves to block 100 to group 60 designate the pixel 16 as being in the second area 20. The 

together changed pixels 16 which are located nearby each pixel 16 is then stored with a tag indicating that it is in the 

other, i.e., which are located within a single nominal bound- second area 20, or it is stored in a bin designated as the 

ary size of each other. The groups could overlap, if desired. second area 20 bin. The process then returns to FIG. 3, block 

For example, one group per changed pixel 16 can be 70. 

established. Alternatively, once the first group has been 65 Now referring to FIG. 6, which would be invoked if timed 

defined, the second group could be defined such that no sampling had been selected by the operator of the system 10 

changed pixels 16 in the first group were in the second at block 70 of FIG. 3, the process starts and, for each pixel 
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16 under test determines at decision 120 whether the pixel 
16 is inside the boundary of the second area 20. This 
determination can be made by accessing the pixel locator 
described above in relation to FIG. 5. If the pixel 16 under 
test is inside the boundary of the second area 20, at block 5 
122 the pixel 16 is saved for processing at block 74 of FIG. 
3, and the process moves to block 124 to retrieve the next 
pixel 16. If any pixels remain to be processed (decision 129), 
the process loops back to decision 120. Otherwise, the 
process returns to FIG. 3, block 72 or 74. 1Q 

If, on the other hand, the pixel 16 under test is not inside 
the boundary of the second area 20 (i.e., it is in the first, 
presumably less important, area 18), it is determined at 
block 126 whether the sampling interval threshold, defined 
either by the operator of the system 10 or the automatic 
method disclosed below in relation to FIG. 7, has been 
reached. In the presently preferred embodiment, the sam- 
pling interval threshold is less than about thirty standard 
video program sampling cycles, i.e., the sampling interval 
threshold is less than about one second. 

20 

If the sampling interval threshold has been reached, the 
process moves to block 128, wherein the pixel 16 is saved 
for processing at block 76 of FIG. 3. The process then moves 
to block 124 to retrieve the next pixel 16 and either loops via 
129, 120, or returns to FIG. 3, block 72 or 74. If, at block 25 
126, the sampling interval threshold has not been reached, 
the process moves to block 130 to apply a null value to the 
pixel 16 under test. Stated differently, the pixel 16 under test 
is suppressed from further processing at block 130. From 
block 130, the process moves to block 124 to retrieve the 3Q 
next pixel 16 and either loops via 129, 120, or returns to FIG. 
3, block 72 or 74. 

Thus, it can now be appreciated that with the sampling 
feature shown in FIG. 6, not only can the pixels 16 in the first 
area 18 be compressed relatively more than the pixels 16 in 35 
the second area 20, but that the compression of the pixels 16 
in the first area 18 occurs less frequently than the compres- 
sion of the pixels 16 in the second area 20. Stated differently, 
the pixels 16 in the first area 18 are processed each sampling 
time interval, whereas the pixels 16 in the second area 20 are 40 
processed each sampling cycle. 

Now referring to FIG. 7, if automatic rate establishment 
had been selected by the operator of the system 10 at block 
82 in FIG. 1 for either one or both of the areas 18, 20, at 
block 132 in FIG. 7 it is determined whether the value of the 45 
pixel 16 under test has changed since the immediately 
preceding image frame. If it has not changed, the next pixel 
16 is retrieved at block 134. If the value of the pixel 16 under 
test has changed, then the pixel 16 is added to a list of 
changed pixels at block 136. 50 

Next, at block 138, it is determined whether the pixel 16 
under test is the last pixel in the area 18, 20 for which a 
sampling interval threshold is being defined. If it is, the 
process moves to block 134. Otherwise, the process moves 
to block 140, wherein the ratio of the number of changed 55 
pixels 16 to unchanged pixels 16 in the particular area 18, 20 
under test is determined. Then, at block 142, a sampling 
interval threshold is determined based upon the ratio deter- 
mined in block 140. This determination can be done by table 
look-up or other appropriate method. go 

From block 142, the process moves to block 144 to 
determine whether the area 18, 20 is the last area for which 
a sampling interval threshold is to be determined. If it is not, 
the process moves to block 134 to retrieve the first pixel 16 
in the next area for which a sampling interval threshold is to 65 
be determined. Otherwise, the process returns to FIG. 3, 
block 86. 



While the particular apparatus and method for processing 
video data as herein shown and described in detail is fully 
capable of attaining the above -described objects of the 
invention, it is to be understood that it is the presently 
preferred embodiment of the present invention and is thus 
representative of the subject matter which is broadly con- 
templated by the present invention, that the scope of the 
present invention fully encompasses other embodiments 
which may become obvious to those skilled in the art, and 
that the scope of the present invention is accordingly to be 
limited by nothing other than the appended claims. 

We claim: 

1. A computer system for processing a video image, 
comprising: 

a central processing unit (CPU); 

a data record having a plurality of elements representing 
a plurality of pixels arranged in a matrix, the data 
record being accessible by the CPU; 

a boundary definer for establishing first and second areas 
of the matrix in response to operator input identifying 
the first and second areas, the boundary definer also for 
inserting into the data record, codes which indicate 
whether the pixels associated with the elements that 
follow reside in the first or second area; 

a locator operably engaged with the boundary definer for 
determining, for at least some of the pixels, which of 
the pixels are located in the first area and which of the 
pixels are located in the second area; and 

a pixel processor operably engaged with the locator for 
processing the elements representing the pixels in a row 
order of the matrix such that elements representing 
pixels in the first area are processed in accordance with 
a first processing scheme and elements representing the 
pixels in the second area are processed in accordance 
with a second processing scheme. 

2. The system of claim 1, wherein the pixel processor 
generates a bit stream representative of the pixels in the row 
order and the area of each pixel. 

3. The system of claim 2, further comprising an I/O device 
for selectively causing the bit stream to be stored or trans- 
mitted. 

4. The system of claim 3, wherein the bit stream repre- 
senting the pixels in the first area is compressed relative to 
the number of pixels in the first area. 

5. The system of claim 1, wherein a video program defines 
a sampling cycle, and the system further comprises a timer 
for establishing a sampling interval threshold longer than the 
sampling cycle and the pixel processor processes the ele- 
ments representing the pixels in the second area each 
sampling cycle, and the pixel processor processes the ele- 
ments representing the pixels in the first area each sampling 
interval threshold. 

6. The system of claim 5, wherein each pixel defines a 
corresponding variable value, and the boundary definer 
establishes the first and second areas in response to varia- 
tions in the values of the pixels. 

7. An apparatus for processing a matrix of data elements 
representing an image, comprising: 

means for establishing first and second areas of the image; 

means for receiving input from an operator indicating 
which of the elements are located in the first area and 
which of the elements are located in the second area; 

means responsive to the input received from the operator 
for inserting into the matrix of data elements, codes 
which indicate in which area the pixels represented by 
the data elements that follow reside; and 



12/23/2003, EAST Version: 1.4.1 



US 6,501,853 Bl 



11 



12 



means for processing the elements in a row order of the 
matrix such that elements in the first area are processed 
in accordance with a first processing scheme and ele- 
ments in the second area are processed in accordance 
with a second processing scheme. 

8. The apparatus of claim 7, wherein the means for 
processing generates a bit stream representative of the 
elements in the row order and the area of each, the apparatus 
further comprises means for selectively causing the bit 
stream to be stored or transmitted, and the bit stream 
representing the elements in the first area is compressed 
relative to the number of elements in the first area. 

9. The apparatus of claim 8, further comprising means for 
establishing a sampling cycle and a sampling interval 
threshold, wherein the sampling interval threshold is longer 
than the sampling cycle and the means for processing 
processes the elements in the first area each sampling cycle, 
and the means for processing processes the elements in the 
second area each sampling interval threshold. 

10. The apparatus of claim 9, wherein each element 
defines a corresponding variable value, and the means for 
establishing establishes the first and second areas in 
response to variations in the values of the elements. 

11. A method for processing data representing pixels 
arranged in a matrix, comprising the steps of: 

(a) defining at least first and second areas within the 
matrix; 

(b) determining which pixels are within the first area and 
which pixels are within the second area in response to 
operator input; 

(c) inserting into the data representing the pixels, codes 
which indicate in which area the pixels represented by 
the data that follows reside; 

(d) processing the data representing pixels in the first area 
in accordance with a first processing scheme; 

(e) processing the data representing pixels in the second 
area in accordance with a second processing scheme; 
and 

(f) performing steps (d) and (e) according to a row order 
of the matrix. 

12. The method of claim 11, further comprising the step 

of: 

(f) establishing a sampling cycle and a sampling interval 
threshold longer than the sampling cycle; and 

(g) processing the data representing pixels in the second 
area each sampling cycle and processing the data 
representing pixels in the first area each sampling 
interval threshold. 

13. The method of claim 12, wherein each pixel defines a 
corresponding variable value, and the method further 
includes establishing the first and second areas in response 
to variations in the values of the pixels. 

14. A system for processing pixels of a video image 
defining a sampling cycle, comprising: 

a data record including a matrix of pixels representing an 
image, the matrix being defined by at least first and 
second areas, the data record further including codes 
inserted in the matrix of pixels in response to operator 
input to indicate in which area the pixels that follow 
reside; 

a timer for establishing a sampling interval threshold 
longer than the sampling cycle; 
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a pixel processor for processing the pixels in a row order 
of the matrix, in which pixels in the first area are 
processed each sampling cycle and pixels in the second 
area are processed each sampling interval threshold. 
5 15. The system of claim 14, wherein each pixel defines a 
corresponding variable value, and the system further com- 
prises a boundary definer for establishing the first and 
second areas in response to variations in the values of the 
pixels. 

10 16. A method for processing pixels, comprising: 

(a) providing a data record including a matrix of pixels in 
row order representing an image, the matrix being 
defined by at least first and second areas and having 
codes inserted therein in response to operator input to 

15 indicate in which area the pixels that follow reside; 

(b) establishing a sampling cycle and a sampling interval 
threshold, wherein the sampling interval threshold is 
longer than the sampling cycle; 

(c) processing the pixels in the first area each sampling 
cycle; and 

(d) processing the pixels in the second area each sampling 
interval threshold. 

17. The method of claim 16, wherein each pixel defines a 
25 corresponding variable value, and the method further com- 
prises: 

(e) establishing the first and second areas in response to 
variations in the values of the pixels. 

18. A method for processing pixels arranged in a two- 
30 dimensional matrix, comprising: 

establishing first and second areas of the matrix; 
receiving operator input to indicate which of the pixels are 

located in the first area and which of the pixels are 

located in the second area; 
inserting into the matrix, codes that indicate in which area 

the pixels that follow reside, in response to the input 

received from the operator; and 
processing the pixels in a row order of the matrix such that 
40 pixels in the first area are processed in accordance with 

a first processing scheme and pixels in the second area 

are processed in accordance with a second processing 

scheme. 

19. The method of claim 18, further comprising: 

45 generating a bit stream representative of the pixels in the 
row order and the area of each pixel; 
selectively causing the bit stream to be stored or trans- 
mitted; and 

compressing the bit stream representing the pixels in the 
first area. 

20. The method of claim 19, further comprising: 
establishing a sampling cycle and a sampling interval 

threshold longer than the sampling cycle; 
processing the pixels in the second area each sampling 
cycle; and 

processing the pixels in the first area each sampling 
interval threshold. 

21. The method of claim 20, wherein each pixel defines a 
60 corresponding variable value, and the first and second areas 

are established in response to variations in the values of the 
pixels. 
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